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(54) Removing chroma noise from digital images by using variable shape pixel neighborhood 
regions 



(57) A method for removing noise on a pixel by pixel 
basis from pixels of a digital image is disclosed. The pix- 
els are used to produce a map of features, which then 
are used to produce a variable shape neighborhood 
region of cleaning pixels with respect to the original 
value of the pixel of interest. The neighborhood region 
of cleaning pixels and the value of the pixel of interest 
are then used to change the original value of the pixel of 
interest in the digital image so that it has been noise 
cleaned. 
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Description 

[0001] The present invention relates to providing smoothed digital images with reduced noise. 
[0002] One type of noise found in digital camera images appears as low frequency, colored blobs in regions of low 
5 spatial frequency, for example, a person's face. These blobs, a type of chroma noise, produce a mottled appearance in 
an otherwise spatially flat region. These colored blobs are irregularly shaped and are typically 5 to 25, or more, pixels 
wide in a given direction. 

[0003] There are numerous ways in the prior art for reducing chroma noise in digital images. Among these are 
numerous patents that describe chroma noise reduction methods using optical blur filters in digital cameras to avoid 
10 aliasing induced chroma noise in the first place. However, these blur filters generally address only high frequency 
chroma noise, and are generally ineffective against low frequency chroma noise. 

[0004] Another very common approach to chroma noise reduction is to simply use standard grayscale image noise 
reduction techniques on each color channel of the image, in effect, treating each color channel as a separate grayscale 
image. By treating a full-color image as three, unrelated grayscale images, any interactions or correlations between the 
75 color channels are ignored. As discussed below, the inherent relationships between the color planes of a digital image 
can be used to perform more effective chroma noise cleaning, for example, by transforming the image into a different 
color space that permits for easier separation of image noise from genuine scene content. 

[0005] Some approaches deal specifically with digital image processing methods for reducing or removing chroma 
noise artifacts. One class of digital camera patents discloses improvements to the color filter array (CFA) interpolation 
20 operation to reduce or eliminate high frequency chroma noise artifacts. Another class of patents teach using different 
pixel shapes (that is, rectangles instead of squares) and arrangements (for example, each row is offset by half a pixel 
width from the preceding row) with accompanying CFA interpolation operations to reduce or eliminate chroma noise 
artifacts. Again, these techniques address only high frequency chroma noise, and are generally ineffective against low 
frequency chroma noise. 

25 [0006] There is the well known technique in the open literature of taking a digital image with chroma noise artifacts, 
converting the image to a luminance — chrominance space, such as CIELAB, blurring the chrominance channels and 
then converting the image back to the original color space. This operation is a standard technique used to combat 
chroma noise. One liability with this approach is that there is no discrimination during the blurring step between chroma 
noise artifacts and genuine chroma scene detail. Consequently, sharp colored edges in the image begin to bleed color 

30 as the blurring become more aggressive. Usually, the color bleed has become unacceptable before most of the low fre- 
quency, colored blobs are removed from the image. Also, if any subsequent image processing is performed on the 
image, there is the possibility of amplifying the visibility of the color bleeding. A second liability of this approach is that 
a small, fixed blur kernel is almost required to try to contain the problem of color bleeding. However, to address low fre- 
quency chroma blobs, large blur kernels would be needed to achieve the desired noise cleaning. 

35 [0007] It is an object of the present invention to provide a chroma noise reduction method that permits for the use 
of large blur kernels while not causing color bleeding at sharp colored edges. 

[0008] It is another object of this invention to provide an improved chroma noise cleaned digital image using varia- 
ble shaped pixel neighborhood region blur kernels. 

[0009] It is another object of the present invention to provide low frequency chroma blobs that can be removed from 
40 a digital image by using variable shaped pixel neighborhood region blur kernels. 

[0010] These objects are achieved with a method for removing noise on a pixel by pixel basis from pixels of a digital 
image comprising the steps of: 

producing a map of features in the digital image; 
45 storing an original value of the pixel of interest from the map of features; 

using values of features from the map to determine a variable shape neighborhood region of cleaning pixels with 
respect to the original value of the pixel of interest; 

using the neighborhood region of cleaning pixels and the value of the pixel of interest to change the original value 
of the pixel of interest in the digital image so that it has been noise cleaned; and 
so repeating the steps above for other pixels of interest. 

[0011] The present invention overcomes the limitation of the "chroma blur trick 11 by first identifying all of the edges 
and boundaries in the image and then permitting each calculation neighborhood region to adaptively grow until it 
encounters an edge or boundary. Consequently, color bleed is avoided while still permitting the use of large area blur- 
55 ring operations to eliminate chroma noise artifacts. 
[001 2] Features of this invention include: 

1) automated operation (no user intervention is required, although the user could be given access to some algo- 
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rithm parameters to control the aggressiveness of image modification); and 

2) locally adaptive, variable sized calculation neighborhood region (very low spatial frequency artifacts can be elim- 
inated in flat regions without forcing the use of the same large calculation neighborhood region size in spatially busy 
areas). 

5 

[0013] A novel aspect of this invention is the use of a locally adaptive, variable sized calculation neighborhood 
region that keys off of an edge feature map to permit the maximum amount of chroma noise removal without significant 
degradation to genuine scene detail. 

io FIG. 1 is a block diagram of the processing flow for practicing the present invention to produce improved digital 
images; and 

FIG. 2 is a typical variable shape pixel neighborhood region employed by this invention. 

[0014] . Since noise cleaning algorithms for digital images are well known, the present description will be directed in 
is particular to elements forming part of, or cooperating more directly with, apparatus and method in accordance with the 
present invention. Elements not specifically shown or described herein may be selected from those known in the art. 
Although the present invention will be described in relation to a digital image produced by an electronic camera, it will 
be understood by those skilled in the art that the present invention is not limited to such image product devices and can 
use original digital images from other sources. 
20 [0015] Referring to FIG. 1, block 10 is the original image. In block 12 the image is converted to a luminance — 
chrominance space, such as CIELAB. Although CIELAB is the preferred color space, other color spaces that can be 
used in accordance with the present invention will suggest themselves to those skilled in the art. Blocks 14, 16, and 18 
create a map of features, or edge map, from the luminance - chrominance data. To create the map, four edge detector 
filters are convolved with each channel and the results summed to create the edge map. The four filters are n h" for hor- 
25 izontal, V for vertical, B s" for slash and n b" for backslash. If the channel image is f(x) and the resulting edge map chan- 
nel is g(x), then we could say: 

g(x) = lh**f(x)l + lv*f (x)l + ls**f(x)l + lb* w f(x)l 

30 where "x" is either the luminance channel, L*, or one of the chrominance channels, a* or b* f is the two-dimensional 
convolution operation, and absolute values of the components are added together. The four edge detector kernels are 
5x5 truncated pyramid filters that were chosen to provide some robustness when used with noisy data. Larger kernels 
could be used for even more noise suppression. The kernels used here are: 
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[0016] In block 20, the final composite edge map, "g", is created by summing together the three channel edge 
55 maps. 

g = g(L*) + g(a*) + g(b*) 
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Since the individual components are already all positive values, there is not need to use additional absolute value oper- 
ations in conjunction with this summation. 

[0017] From the features edge map, a threshold value is determined. Typically, for an initial estimate, one chooses 
a flat region of the image and calculates the standard deviation (stdv) of the edge map values in this region. Three times 

5 the standard deviation (3 stdv) becomes the threshold. If this algorithm is being run interactively, this threshold could be 
one of the knobs given to the user. If it is found that for a given application the threshold does not significantly change 
from image to image, the threshold can be determined once, off-line, and then left as a fixed value for future use. 
[0018] The chrominance channels, a* and b* ( are now smoothed. In other words they are noise cleaned. In block 
22, for each pixel In the image, the corresponding edge map value is taken to be the reference value. Then the algorithm 

10 moves out in each of the eight compass directions, N, NE, E, SE, S, SW, W, NW, one pixel at a time, examining the edge 
map values. If the difference between an edge map value and the reference value is less than the threshold, that pixel 
is added to the smoothing neighborhood region and the algorithm continues. Once an edge map value is reached that 
differs from the reference value by more than the threshold, the growth of the smoothing neighborhood region in that 
compass direction is stopped. FIG. 2 illustrates a typical smoothing neighborhood region after all compass directions 

15 have been examined. In this arrangement, eight directions are shown since there are eight contiguous pixels surround- 
ing the pixel of interest. However, a contiguous neighborhood region is not a requirement of this invention. Note that this 
neighborhood region will be variable in shape and size. Returning to FIG. 1 , within the smoothing neighborhood region, 
in block 24, the a* and b* channel values are averaged and these averages replace the a* and b* channel values for 
pixel A in FIG. 2. Each pixel in the image is processed in this way. In block 26, FIG. 1 , the final image is converted back 

20 to the original color space. 

[0019] The maximum possible direction of neighborhood region expansion in any given compass direction can be 
restricted to prevent huge neighborhood regions from being used in large flat regions. Such a maximum radius value 
might be in the range from 50 to 100 pixels. 

[0020] The present invention can be embodied in a software program provided in a computer program product, hav- 
25 ing a computer readable storage medium having a computer program stored thereon for removing noise on a pixel by 
pixel basis from pixels of a digital image. The computer program performs the steps of: 



(a) producing a map of features in the digital- image; 

(b) storing an original value of the pixel of interest from the map of features; 

30 (c) using values of features from the map to determine a variable shape neighborhood region of cleaning pixels with 

respect to the original value of the pixel of interest; 

(d) using the neighborhood region of cleaning pixels and the value of the pixel of interest to change the original 
value of the pixel of interest in the digital image so that it has been noise cleaned; and 

(e) repeating steps (a)-(d) for other pixels of interest. 

35 

[0021] The computer readable storage medium may comprise, for example, magnetic storage media, such as mag- 
netic disks (such as a floppy disk) or magnetic tape; optical storage media such as an optical disk, optical tape or 
machine readable memory (RAM), or read only memory (ROM); or any other physical device or medium. 

40 Claims 



1. A method for removing noise on a pixel by pixel basis from pixels of a digital image comprising the steps of: 

(a) producing a map of features in the digital image; 
45 (b) storing an original value of the pixel of interest from the map of features; 

(c) using values of features from the map to determine a variable shape neighborhood region of cleaning pixels 
with respect to the original value of the pixel of interest; 

(d) using the neighborhood region of cleaning pixels and the value of the pixel of interest to change the original 
value of the pixel of interest in the digital image so that it has been noise cleaned; and 

so (e) repeating steps (a)-(d) for other pixels of interest. 



2. The method of claim 1 wherein the neighboring contiguous pixels are disposed along predetermined directions 
which pass through the pixel of interest and the extent of the pixels along each direction are determined by the 
value of the features along such direction compared to the original pixel value. 

55 

3. The method of claim 1 wherein the neighboring contiguous pixels are disposed along predetermined directions 
which pass through the pixel of interest and the extent of the pixels along each direction are determined by the 
value of an edge along such direction. 
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4. A computer program product, comprising: 

a computer readable storage medium having a computer program stored thereon for removing noise on a pixel 
by pixel basis from pixels of a digital image for performing the steps of: 

5 

(a) producing a map of features in the digital image; 

(b) storing an original value of the pixel of interest from the map of features; 

(c) using values of features from the map to determine a variable shape neighborhood region of cleaning 
pixels with respect to the original value of the pixel of interest; 

10 (d) using the neighborhood region of cleaning pixels and the value of the pixel of interest to change the 

original value of the pixel of interest in the digital image so that it has been noise cleaned; and 
(e) repeating steps (a) -(d) for other pixels of interest. 
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